{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# ERNIE Models Dialogue Application Demo\n",
    "\n",
    "\n",
    "This tutorial demonstrates how to use the ERNIE models to implement a simple demo for dialogue application. The following two models are mainly used:\n",
    "- Text-only Model: Supports text-based dialogue interactions.\n",
    "- Multimodal Model: Enables mixed input of both images and text for dialogue scenarios."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1. Environmental Setup\n",
    "Before starting, ensure your system meets these requirements:\n",
    "- Python version 3.10-3.12 is installed.\n",
    "- Ensure the following Python libraries are included: `openai`, `base64`\n",
    "- Set the API key for the model `model_api_key`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "model_api_key = \"api_key\" # the API key for model, which can be disregarded and replaced with any arbitrary value when using the model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1.1. Install Dependencies"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!pip install openai"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2. Text-Only Model\n",
    "The text-only model is ideal for basic language tasks that involve processing text input and generating corresponding output (e.g., question-answering, summarization, translation), without requiring complex reasoning capabilities.\n",
    "\n",
    "When using a text-only model, you need to deploy the [ERNIE-4.5](https://github.com/PaddlePaddle/FastDeploy) series model services and correctly configure the corresponding service address `ernie_45_url`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ernie_45_url = \"http://localhost:port/v1\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.1. Model Input\n",
    "The model input consists of a message list that captures the conversational context history. Each message in the list is structured as a dictionary and includes the following fields:\n",
    "- `role`: the role of the message sender, which can be:\n",
    "    - `system`: System message, usually used to set the behavior of the model or initial instructions\n",
    "    - `user`: User message, indicating user inputs\n",
    "    - `assistant`: Model message, indicating the model's reply\n",
    "- `content`: Specific text content\n",
    "\n",
    "Text-only model input has the following characteristics:\n",
    "- System Role: System prompt words can be set to define the behavior of the model\n",
    "- Text-Only Dialogue: only text input and output are supported\n",
    "- Multiple Rounds of Dialogue: Supporting the preservation of historical dialogue context"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "text_messages = [\n",
    "    {\"role\": \"system\", \"content\": \"你是一个智能助手\"},\n",
    "    {\"role\": \"user\", \"content\": \"中国的首都是哪里？\"},\n",
    "    {\"role\": \"assistant\", \"content\": \"北京\"},\n",
    "    {\"role\": \"user\", \"content\": \"北京的地铁怎么样？\"}\n",
    "]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.2. Non-Streaming Request\n",
    "#### 2.2.1. Request Model\n",
    "When sending a request to the API, the following main parameters need to be considered:\n",
    "- `messages` (must): List of text conversation messages\n",
    "- `max_tokens` (optional): configuration parameter for maximum number of generated tokens\n",
    "- `temperature` (optional): configuration parameter for controlling randomness in generated results\n",
    "- `top_p` (optional): configuration parameter for nucleus sampling"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'id': 'chatcmpl-fde2edb1-1b0d-4a59-b0e5-3918916f6c20', 'choices': [{'finish_reason': 'stop', 'index': 0, 'logprobs': None, 'message': {'content': '北京地铁的票价贵吗？\\n\\n北京地铁非常方便，覆盖面广，可以方便地到达城市的各个角落。票价方面，北京地铁的票价相对来说比较合理，根据乘坐的距离长短来计算票价，起步价为3元，根据行程长度进行计价，最高票价一般为10元。同时，北京地铁也提供一些优惠政策，如学生票、老年人票等，让更多人能够享受公共交通的便利。</s></s>', 'refusal': None, 'role': 'assistant', 'annotations': None, 'audio': None, 'function_call': None, 'tool_calls': None, 'reasoning_content': None}}], 'created': 1750076721, 'model': 'default', 'object': 'chat.completion', 'service_tier': None, 'system_fingerprint': None, 'usage': {'completion_tokens': 91, 'prompt_tokens': 29, 'total_tokens': 120, 'completion_tokens_details': None, 'prompt_tokens_details': {'audio_tokens': None, 'cached_tokens': 0}}}\n"
     ]
    }
   ],
   "source": [
    "from openai import OpenAI\n",
    "\n",
    "client = OpenAI(base_url=ernie_45_url, api_key=model_api_key)\n",
    "response_text = client.chat.completions.create(\n",
    "    model=\"default\",\n",
    "    messages=text_messages,\n",
    "    temperature=1.0,\n",
    "    max_tokens=2048,\n",
    "    top_p=0.7\n",
    ")\n",
    "response_text = response_text.model_dump()\n",
    "print(response_text)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 2.2.2 Model Output\n",
    "- `content`: Final answer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "北京的地铁非常发达，是世界上最长的地铁系统之一。它覆盖了城市中心和远郊区域，为市民和游客提供了方便快捷的交通方式。以下是一些北京地铁的特点：\n",
      "\n",
      "1. 线路覆盖面广：北京地铁已经开通了多条线路，覆盖了城市的主要区域，包括市中心、近郊和远郊。\n",
      "\n",
      "2. 运营时间长：北京地铁的运营时间通常是从早上5点到晚上11点左右，具体根据不同线路有所调整。\n",
      "\n",
      "3. 客流量大：由于北京是一个人口众多的城市，地铁客流量也非常大。在高峰时段，一些线路会非常拥挤。\n",
      "\n",
      "4. 技术先进：北京地铁采用了先进的信号技术和自动控制系统，确保了列车的安全和准时。\n",
      "\n",
      "5. 票价便宜：北京地铁的票价相对便宜，是根据乘坐的站数来计算的，最高票价也不会超过10元。\n",
      "\n",
      "总的来说，北京地铁是一种非常方便、快捷、经济实惠的交通方式，为市民和游客提供了良好的出行体验。</s></s>\n"
     ]
    }
   ],
   "source": [
    "content_text = response_text[\"choices\"][0][\"message\"][\"content\"]\n",
    "print(content_text)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2.3. Streaming Request\n",
    "#### 2.3.1. Request Model\n",
    "- `messages` (must): List of conversation messages\n",
    "- `max_tokens` (optional): configuration parameter for maximum number of generated tokens\n",
    "- `temperature` (optional): configuration parameter for controlling randomness in generated results\n",
    "- `top_p` (optional): configuration parameter for nucleus sampling\n",
    "- `stream` (optional): configuration parameter for enabling/disabling streamed return"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[{'id': 'chatcmpl-4b334e11-bb61-46fe-92e7-f3db8799ab5f', 'choices': [{'delta': {'content': '', 'function_call': None, 'refusal': None, 'role': 'assistant', 'tool_calls': None, 'reasoning_content': ''}, 'finish_reason': None, 'index': 0, 'logprobs': None}], 'created': 1750076763, 'model': 'default', 'object': 'chat.completion.chunk', 'service_tier': None, 'system_fingerprint': None, 'usage': None}, {'id': 'chatcmpl-4b334e11-bb61-46fe-92e7-f3db8799ab5f', 'choices': [{'delta': {'content': '北京', 'function_call': None, 'refusal': None, 'role': None, 'tool_calls': None, 'token_ids': [3991], 'reasoning_content': None}, 'finish_reason': None, 'index': 0, 'logprobs': None, 'arrival_time': 0.058094024658203125}], 'created': 1750076763, 'model': 'default', 'object': 'chat.completion.chunk', 'service_tier': None, 'system_fingerprint': None, 'usage': None}, {'id': 'chatcmpl-4b334e11-bb61-46fe-92e7-f3db8799ab5f', 'choices': [{'delta': {'content': '地铁', 'function_call': None, 'refusal': None, 'role': None, 'tool_calls': None, 'token_ids': [15560], 'reasoning_content': None}, 'finish_reason': None, 'index': 0, 'logprobs': None, 'arrival_time': 0.08609342575073242}], 'created': 1750076763, 'model': 'default', 'object': 'chat.completion.chunk', 'service_tier': None, 'system_fingerprint': None, 'usage': None}]\n"
     ]
    }
   ],
   "source": [
    "client = OpenAI(base_url=ernie_45_url, api_key=model_api_key)\n",
    "response_text = client.chat.completions.create(\n",
    "    model=\"default\",\n",
    "    messages=text_messages,\n",
    "    temperature=1.0,\n",
    "    max_tokens=2048,\n",
    "    top_p=0.7,\n",
    "    stream=True\n",
    ")\n",
    "response_text_stream = []\n",
    "for chunk in response_text:\n",
    "    if not chunk.choices:\n",
    "        continue\n",
    "    response_text_stream.append(chunk.model_dump())\n",
    "\n",
    "print(response_text_stream[:3])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 2.3.2. Model Output\n",
    "The model's output will be delivered via streaming return.\n",
    "- `content`: Final answer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "北京地铁的现状和发展情况如何？\n",
      "\n",
      "北京地铁是一项庞大而复杂的交通系统，为北京市民和游客提供了便捷的城市交通方式。以下是一些关于北京地铁的现状和发展情况：\n",
      "\n",
      "1. 线路覆盖范围：北京地铁目前拥有多条线路，覆盖了城市的主要区域和重要景点。不仅如此，地铁线路还在不断扩展，以满足城市发展的需求。\n",
      "\n",
      "2. 运营情况：北京地铁每天运营时间较长，通常从早上5点到晚上11点左右，高峰期间会加强列车频次以应对乘客需求。地铁列车在繁忙时段可能会非常拥挤，但运营方也在努力提高运力和舒适度。\n",
      "\n",
      "3. 技术创新：北京地铁引入了一些先进的技术，如智能支付系统、无线网络覆盖、安全监控系统等，以提升乘客体验和运营效率。\n",
      "\n",
      "4. 环保可持续发展：北京地铁也在积极推动环保和可持续发展，采用节能技术、噪音控制措施等，减少对环境的影响。\n",
      "\n",
      "5. 未来规划：北京地铁系统有进一步的扩展规划，包括新线路的建设和既有线路的改造升级，以适应城市发展和人口增长的需求。\n",
      "\n",
      "总体来说，北京地铁在不断发展壮大，为城市居民和游客提供了便捷、高效的交通选择。</s>\n"
     ]
    }
   ],
   "source": [
    "content_text_stream = \"\"\n",
    "for res in response_text_stream:\n",
    "    content_text_stream += res[\"choices\"][0][\"delta\"][\"content\"]\n",
    "print(content_text_stream)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3. Multimodal Model\n",
    "The multimodal model is well-suited for cross-modal task scenarios that demand the simultaneous processing of visual and textual information, including applications such as image captioning, visual-textual question answering (VQA), and other tasks requiring comprehensive visual semantic comprehension.\n",
    "\n",
    "When using multimodal models, you need to deploy the [ERNIE-4.5-VL](https://github.com/PaddlePaddle/FastDeploy) series model services and correctly configure the corresponding service address `ernie_45_vl_url`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ernie_45_vl_url = \"http://localhost:port/v1\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.1. Model Input\n",
    "Multimodal model input has the following characteristics:\n",
    "- System Role: System prompt words can be set to define the behavior of the model\n",
    "- Multiple Rounds of Dialogue: Supporting the preservation of historical dialogue context\n",
    "- Mixed Input: Supports sending pictures and text content at the same time\n",
    "- Image Support: You can upload pictures through URL or local files"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "# url pictures\n",
    "multi_messages_url = [\n",
    "    {\"role\": \"system\", \"content\": \"你是一个图片分析助手\"},\n",
    "    {\"role\": \"user\", \"content\": [\n",
    "        {\"type\": \"image_url\", \"image_url\": {\"url\": \"https://nlp-eb.cdn.bcebos.com/static/eb/asset/topLogo.4a0fc7b7\"}},\n",
    "        {\"type\": \"text\", \"text\": \"请分析这张图\"}\n",
    "    ]}\n",
    "]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "When uploading local pictures, you need to use the `base64` library to convert the image file into a Base64 encoded string and splice it into a standard format of `data:image/{type};base64,{Base64 encoded string}`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[{'role': 'system', 'content': '你是一个图片分析助手'}, {'role': 'user', 'content': [{'type': 'image_url', 'image_url': {'url': ''}}, {'type': 'text', 'text': '请分析这张图'}]}]\n"
     ]
    }
   ],
   "source": [
    "import base64\n",
    "\n",
    "\n",
    "# Upload pictures locally\n",
    "def get_image_url(image_path):\n",
    "    base64_image = \"\"\n",
    "    extension = image_path.split(\".\")[-1]\n",
    "    with open(image_path, \"rb\") as image_file:\n",
    "        base64_image = base64.b64encode(image_file.read()).decode(\"utf-8\")\n",
    "    url = f\"data:image/{extension};base64,{base64_image}\"\n",
    "    return url\n",
    "\n",
    "test_image = \"../assets/yiyan_logo.png\"\n",
    "image_url = get_image_url(test_image)\n",
    "multi_messages_local = [\n",
    "    {\"role\": \"system\", \"content\": \"你是一个图片分析助手\"},\n",
    "    {\"role\": \"user\", \"content\": [\n",
    "        {\"type\": \"image_url\", \"image_url\": {\"url\": image_url}},\n",
    "        {\"type\": \"text\", \"text\": \"请分析这张图\"}\n",
    "    ]}\n",
    "]\n",
    "print(multi_messages_local)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.2. Request Model\n",
    "When sending a request to the API, the following main parameters need to be considered:\n",
    "- `messages` (must): a list of graphic and text dialogue messages. The pictures can be uploaded locally (`multi_messages_local`), or you can enter the URL address (`multi_messages_url`)\n",
    "- `max_tokens` (optional): configuration parameter for maximum number of generated tokens\n",
    "- `temperature` (optional): configuration parameter for controlling randomness in generated results\n",
    "- `top_p` (optional): configuration parameter for nucleus sampling"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'id': 'chatcmpl-a3f63b46-ab8a-4d7e-a62c-27709dfac710', 'choices': [{'finish_reason': 'stop', 'index': 0, 'logprobs': None, 'message': {'content': '这是一张“文心一言”的logo图片。\\n### 图形部分\\n左侧图形整体是六边形轮廓，内部有类似信息流动或智能交互的抽象图案，采用蓝色调，蓝色常给人科技、专业、理智的感觉，这与产品作为人工智能语言模型的科技属性相契合。\\n### 文字部分\\n右侧“文心一言”四个汉字，字体设计简洁现代，同样使用蓝色，与左侧图形在色彩上相呼应，整体视觉风格统一，突出了产品的品牌名称，便于用户识别和记忆。 \\n\\n该logo通过简洁的设计和具有科技感的元素，传达出产品智能化、现代化的特点。</s></s>', 'refusal': None, 'role': 'assistant', 'annotations': None, 'audio': None, 'function_call': None, 'tool_calls': None, 'reasoning_content': None}}], 'created': 1750077575, 'model': 'default', 'object': 'chat.completion', 'service_tier': None, 'system_fingerprint': None, 'usage': {'completion_tokens': 135, 'prompt_tokens': 54, 'total_tokens': 189, 'completion_tokens_details': None, 'prompt_tokens_details': {'audio_tokens': None, 'cached_tokens': 0}}}\n"
     ]
    }
   ],
   "source": [
    "from openai import OpenAI\n",
    "\n",
    "client = OpenAI(base_url=ernie_45_vl_url, api_key=model_api_key)\n",
    "response_multi = client.chat.completions.create(\n",
    "    model=\"default\",\n",
    "    messages=multi_messages_local,\n",
    "    temperature=1.0,\n",
    "    max_tokens=2048,\n",
    "    top_p=0.7\n",
    ")\n",
    "response_multi = response_multi.model_dump()\n",
    "print(response_multi)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.3. Model Output\n",
    "- `content`: Final answer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "这是一张“文心一言”的logo图片。\n",
      "### 图形部分\n",
      "左侧图形整体是六边形轮廓，内部有类似信息流动或智能交互的抽象图案，采用蓝色调，蓝色常给人科技、专业、理智的感觉，这与产品作为人工智能语言模型的科技属性相契合。\n",
      "### 文字部分\n",
      "右侧“文心一言”四个汉字，字体设计简洁现代，同样使用蓝色，与左侧图形在色彩上相呼应，整体视觉风格统一，突出了产品的品牌名称，便于用户识别和记忆。 \n",
      "\n",
      "该logo通过简洁的设计和具有科技感的元素，传达出产品智能化、现代化的特点。</s></s>\n"
     ]
    }
   ],
   "source": [
    "content_multi = response_multi[\"choices\"][0][\"message\"][\"content\"]\n",
    "print(content_multi)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
